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SELECTIVE CONVERSATION RECORDING USING SPEECH HEURISTICS 

TECHNICAL FIELD OF THE INVENTION 

The present invention relates to the selective 
handling and recording of telephone conversations. More 
particularly, the present invention relates to utilizing 
speech analysis methods for selectively handling and 
recording telephone conversations. 
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BACKGROUND OF THE INVENTION 

Many corporate telephone call centers record the 
telephone conversations between their telephone operators 
and customers. Typically the recordings are made either 
for training purposes, used as examples to be reviewed 
later by new operators, . or for quality assurance 
purposes, so that the performance of the telephone 
operators can be monitored and evaluated. 

Today call centers may record telephone calls in one 
of two ways: (1) by either randomly recording 
conversations, or (2) by recording all conversations. 
When telephone interactions are screened randomly, a 
strong possibility exists that important conversations 
will not be recorded. In the case . where all calls are 
recorded, a tremendous amount of resources can go to 
waste, since the vast majority of call center 
conversations are likely to be uneventful. Moreover, 
call center systems that are currently in place provide 
no means for alerting a supervisor in real-time if a 
problem situation arises other than by the call center 
agent- Instead, supervisors must rely on a subsequent 
review of the recording. Often times, from a customer 
service perspective, it is too late to rectify the 
situation. 
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SUMMARY OF THE INVENTION 

The present invention relates to a system and method 
for selectively monitoring, recording, storing, and 
handling telephone conversations through the use of 
speech analysis. The system and method of the present 
invention records and saves only those conversations that 
exhibit certain problem signs, providing the technical 
advantage of saving storage space over conventional 
systems and methods. Further, later review of calls 
recorded in accordance with the present invention is less 
cumbersome since every stored call would likely be a 
problem call. This is another technical advantage of the 
present invention. Additionally, the present invention 
provides real-time alerts, e.g. to a supervisor, such 
that a call falling within certain pre-defined parameters 
can be monitored so that a decision can be made as to how 
to best handle the telephone call. This provides the 
technical advantage that call center supervisors have 
greater control over their call operators and a higher 
level of service to customers, in a more efficient manner 
than conventional systems and methods. 

In accordance with an exemplary embodiment of the 
present invention a monitoring and recording device is 
provided which monitors and records telephone 
conversations between the call center operators and the 
outside customers. A speech analyzer is used to monitor 
the communication between an operator and a customer 
during a telephone conversation, while a recording and 
storage device records the conversation. Based on 

variations in frequency and amplitude of the voice 
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signals, the speech analyzer can identify changes in 
emotion of the parties. Once a change in emotion is 
detected which matches a predefined trigger event, the 
monitoring and recording device marks the recording of 
the call and can automatically alert the call operator's 
supervisor or supervisor's station. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a complete understanding of the present 
invention and the advantages thereof, reference is now 
made to the following description taken in conjunction 
5 with the accompanying drawings in which like reference 
numbers indicate like features, components and method 
steps, and wherein: 

FIGURE 1 is an illustration of an exemplary- 
embodiment of the present invention and its placement 
10 within a typical call center; 

FIGURE 2 is an illustration of an exemplary 
embodiment of the present invention; 

FIGURE 3 is a flow chart illustrating the operation 
of an exemplary system in accordance with the present 
15 invention; and 

FIGURE 4 an illustration of an exemplary embodiment 
of the present invention and its placement within, a call 
center which utilizes an Internet Protocol -based 
t e 1 ephone sy s t em , 
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DETAILED DESCRIPTION OF THE INVENTION 

Now referring to the drawings, FIGURE 1 illustrates 
the conventional connection established between a 
customer 100 and a call center 104 during a telephone 
call and an exemplary placement of the present invention 
within a call center 104. The customer 100 is connected 
to a call center 104 through the public telephone network 
102. The public telephone network 102 is commonly known 
to those skilled in the art. The call center 104 may 
include a telephone switch or hub 106, one or more 
operators 108, and one or more monitoring and recording 
devices 110. 

FIGURE 2 illustrates one exemplary embodiment of the 
present invention. The monitoring and recording device 
110 is connected such that it can monitor the speech 
signals from both a customer 10 0 and an operator 108 when 
they are connected in a telephone conversation. The 
monitoring and recording device 110 also includes a 
speech analyzer 200,' controller circuit 202 and a 
recording and storage device 204. The controller circuit 
2 02 can be a general purpose computer, a microcontroller, 
ASIC, FPGA, EEPROM, ROM, RAM, any combination thereof, or 
any other circuit which can be used to control the other 
components of monitoring and recording device 104. 
Alternatively, the functional aspects of the controller 
circuit 202 may be integrated into either the speech 
analyzer 200 or the recording and storage device 304 for 
simplicity of design. The recording and storage device 
2 04 can include any means for recording and storing audio 
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readable memory, hard disk, and magnetic-optical drive. 

In operation, a telephone call is first connected 
between the operator 108 and the customer 100, The call 
5 may be initiated by either the operator 108 or the 
M= customer 100. Using conventional methods, the telephone 

Q switch 106 is used to route a customer call to one or 

more operators 108 within the call center 104, 

lis 

ul FIGURE 3 is a flowchart illustrating the operation 

JVj 10 of an exemplary embodiment of the present invention. 

= Once the telephone call is connected at block 3 00, the 

fy speech analyzer 200 detects the signal on the line and 

JJJ prompts the recording and storage device 2 04 to begin 

D recording the conversation at block 302. This prompt can 

15 either be sent directly by the speech analyzer 200 to the 
recording and storage device 204, or indirectly, by the 
prompting controller circuit 2 02 which then would send a 
command to the recording and storage device 2 04, i.e., 
the controller circuit 2 02 knows that line is connected 

2 0 and prompts the speech analyzer 200 to begin analyzing 

and tells the recording and storage device 204 to start 
recording. At block 3 04, during an initial period 
thereafter, the controller circuit 2 02 and the speech 
analyzer 200 work in conjunction to establish base values 
25 to identify signal characteristics typical for the 
conversation. Once the base values are established, the 
system continues to monitor the conversation for 
deviations in the amplitude and/or frequency values 
falling outside of a threshold range, defined by the base 

3 0 values plus or minus a calculated or predetermined value. 
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At block 3 08, the system determines whether a trigger 
event is detected. If the speech signals never deviate 
outside of the threshold range, i.e., no triggering event 
is detected, once the call terminates in a normal 
5 fashion, the recording of the conversation may be deleted 
at block 308 by the recording and storage device 204. If 
the signals do exceed the threshold range at any point 
during the conversation, i.e., a triggering event is 
detected at block 306, the controller circuit 202 at 
10 block 310 takes appropriate actions, as described more 
fully below. 

In accordance with an exemplary embodiment of the 
present invention, once a triggering event is detected, 
the controller circuit 202 may mark the recording of the 

15 conversation and command the recording and storage device 
204 to save the call for future review. Additionally, 
detection of a triggering event may generate an alert to 
the call center supervisor or supervisor's station. The 
controller circuit 202 transmits a report of the event 

2 0 and related identification data to the supervisor via 
computer network or other some other conventional data 
transmission method. The alert contains information such 
as the identity and location of the call operator 108 and 
the type - of event detected. Moreover, the supervisor 

25 could be given the option to "snoop" the call, which 
would allow the supervisor to listen to the conversation, 
or to take over the call altogether, by transferring the 
call from the operator's telephone to the supervisor's 
station, e.g., telephone. 
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Other embodiments may also be employed through 
modification of the base system of FIG 2. One 
alternative is to change the physical placement of the' 
monitoring and recording device 110 within the call 
5 center 104 . Rather than including one monitoring and 
^ recording device 110 at each operator's station 108, one 

□ alternative is to utilize one multi-port monitoring and 

recording device connected at the call center telephone 
yj switch 106, This multi-port monitoring and recording 

Q 10 device located at the telephone switch 106 has the 

equivalent functionality of a plurality of monitoring and 
rj recording devices 110, and would be capable of monitoring 

g several different lines and recording the data on each at 

H ^ny given time. It would have easy access to all 

15 incoming and outgoing communications at the call center 
104. This centralized placement may provide benefits in 
terms of installation and maintenance of the system. 

FIGURE 4 shows another exemplary embodiment of the 
present invention, where a monitoring and recording 
20 device 406 is used in an Internet Protocol (IP) telephony 
environment. The system is structurally similar to the 
previously described exemplary embodiment of FIGURE 2 . 
In the exemplary embodiment of FIGURE 4, IP server 402 
connects to a plurality, of operator IP telephones 404. 
25 An operator IP telephone 404 can be a stand-alone IP 
telephone device, an IP softphone implemented on a 
general purpose computer, or any other means for IP voice 
communication. Also connected to the server 402 is an IP 
monitoring and recording device 406, or plurality 
30 thereof. The system operates in a similar manner to the 
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monitoring and recording device 110 of FIGURE 2, but 
there are differences in the implementation. In this 
system, there is no single point to which the monitoring 
and recording device 4 06 can be connected in order to 
monitor the conversation. Instead, the monitoring and 
recording device 4 06 may be attached to the network and 
assigned its own IP address. Data traffic from calls 
between an operator and a customer must be routed to the 
IP monitoring and recording device 406. This can be done 
in several ways. One way is to force the operator IP 
telephone device 4 04 to forward all data, both received 
from the customer end and transmitted by the operator IP 
telephone device 4 04, to the IP monitoring and recording 
device 406. This would ensure that the IP monitoring and 
recording device 406 received all relevant data packets 
and therefore a full voice data stream from each party 
for monitoring and storage. 

In another exemplary embodiment, the IP voice data 
could be routed to the IP monitoring and recording device 
406 using an alternative method. Rather than requiring 
the operator telephone device 4 04 to forward the data to 
the IP monitoring and recording device 4 06, the system 
could automatically create a conference call with the IP 
monitoring and recording device 4 06 every time an 
operator connects with an outside customer. The IP 
address of the IP monitoring and recording device 406 
could be included automatically as part of the conference 
call, thereby ensuring that the IP monitoring and 
recording device 4 06 receives the speech data from both 
parties. Once a system is implemented which routes the 
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data to the IP monitoring and recording device 4 06, the 
system would function in much the same way as its analog 
counterpart described above, the only necessary change 
being different equipment used to perform the speech 
analysis and the use of a network protocol, e.g., 
Transmission Control Protocol/Internet Protocol (TCP/IP) . 

Although the present invention has been described in 
detail with reference to specific exemplary embodiments 
thereof, various modifications, alterations and 
adaptations may be made by those skilled in the art 
without ' departing from the spirit and scope of the 
invention. It is intended that the invention be limited 
only by the appended claims. 



